Search for: All records

Abstract We study a class of Approximate Message Passing (AMP) algorithms for symmetric and rectangular spiked random matrix models with orthogonally invariant noise. The AMP iterates have fixed dimension $$K \geq 1$$, a multivariate non-linearity is applied in each AMP iteration, and the algorithm is spectrally initialized with $$K$$ super-critical sample eigenvectors. We derive the forms of the Onsager debiasing coefficients and corresponding AMP state evolution, which depend on the free cumulants of the noise spectral distribution. This extends previous results for such models with $K=1$ and an independent initialization. Applying this approach to Bayesian principal components analysis, we introduce a Bayes-OAMP algorithm that uses as its non-linearity the posterior mean conditional on all preceding AMP iterates. We describe a practical implementation of this algorithm, where all debiasing and state evolution parameters are estimated from the observed data, and we illustrate the accuracy and stability of this approach in simulations. 

Many recent works have studied the eigenvalue spectrum of the Conjugate Kernel (CK) defined by the nonlinear feature map of a feedforward neural network. However, existing results only establish weak convergence of the empirical eigenvalue distribution, and fall short of providing precise quantitative characterizations of the “spike” eigenvalues and eigenvectors that often capture the low-dimensional signal structure of the learning problem. In this work, we characterize these signal eigenvalues and eigenvectors for a nonlinear version of the spiked covariance model, including the CK as a special case. Using this general result, we give a quantitative description of how spiked eigenstructure in the input data propagates through the hidden layers of a neural network with random weights. As a second application, we study a simple regime of representation learning where the weight matrix develops a rank-one signal component over training and characterize the alignment of the target function with the spike eigenvector of the CK on test data. 

Abstract Photometric stellar surveys now cover a large fraction of the sky, probe to fainter magnitudes than large-scale spectroscopic surveys, and are relatively free from the target selection biases often associated with such studies. Photometric-metallicity estimates that include narrow/medium-band filters can achieve comparable accuracy and precision to existing low-resolution spectroscopic surveys such as Sloan Digital Sky Survey/SEGUE and LAMOST. Here we report on an effort to identify likely members of the Galactic disk system among the very metal-poor (VMP; [Fe/H] ≤ −2) and extremely metal-poor (EMP; [Fe/H] ≤ −3) stars. Our analysis is based on an initial sample of ∼11.5 million stars with full space motions selected from the SkyMapper Southern Survey (SMSS) and Stellar Abundance and Galactic Evolution Survey (SAGES). After applying a number of quality cuts to obtain the best available metallicity and dynamical estimates, we analyze a total of ∼5.86 million stars in the combined SMSS/SAGES sample. We employ two techniques that, depending on the method, identify between 876 and 1476 VMP stars (6.9%−11.7% of all VMP stars) and between 40 and 59 EMP stars (12.4%−18.3% of all EMP stars) that appear to be members of the Galactic disk system on highly prograde orbits (v_ϕ> 150 km s⁻¹). The total number of candidate VMP/EMP disklike stars is 1496, the majority of which have low orbital eccentricities, ecc ≤ 0.4; many have ecc ≤ 0.2. The large fractions of VMP/EMP stars associated with the Milky Way disk system strongly suggest the presence of an early-forming “primordial” disk.